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Non-intellective Predictors of Achievement in College 

Robert C. Nichols 

National Merit Scholarship Corporation 

Selection of students in college admissions and scholarship programs is 
usually based in part on a prediction of the student's performance in college. 

The standard predictors are aptitude test scores and high school grades, often 
combined into a single index by a regression equation. The standard criterion 
for validation of these predictions is the college grade point average, usually 
for the freshman year. There is a well developed technology for making these 
predictions, and many colleges and scholarship agencies have developed formulas 
for optimum weighting of high school grades and test scores in their local 
situation. 

The growing uniformity of selection practices might lead one to assume 
that the prediction of college performance has reached a state of stable matur- 
ity satisfactory to most practitioners. Instead, there is increasing concern 
and dissatisfaction with the current state of the art. This concern may be 
in part a reaction to the recently vocal lay critics of the testing movement, 
but there is also a recognition of the inadequacy of current selection methods 
to meet the demands being made of them. There are two major sources of dis- 
satisfaction: 

1. The popular colleges and most scholarship and honors programs have 
many more applicants than they can accept. After applicants with low grades 
or low test scores are eliminated, further discriminations must be made 
between the remaining highly qualified candidates. The selection committee 
can mak e these decisions on the basis of the remaining small differences 
in test scores and high school rank; they can look at other data, such 
as an interviewer's impression, a recommendation or an autobiography; or 
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they can flip a coin. These three methods of discriminating between stu- 
dents with good grades and high test scores are about equally valid when 
judged by the criterion of later student performance. Educational 
researchers have been busy searching for indicators of success other than 
grades and test scores (usually called non-intellective predictors) and 
some have been found. However, none has yet achieved enough demonstrated 
success to be widely adopted. 

2. There is growing dissatisfaction with the use of college grades as 
the criterion for evaluation of predictors. Leading educators are begin- 
ning to feel that the student who makes the best grades is not necessarily 
the most valuable student. For example, Stalnaker (1965) in discussing 
the National Merit Scholarship selection program, said, ". . .we want to 

find students who will succeed in college, but much more important 

will also use their college education in some socially desirable, pro- 
ductive way after graduation. How relevant are grades to this goal?. . . 
Do you inquire of your accountant, your physician or your lawyer the 
grades he received in college? Predicting grades has little social sig- 
nificance" (p. 134). Holland and Richards (1965) have pointed out that 
a student's extracurricular achievement may be more similar to achieve- 
ment after graduation than is the academic achievement represented by 
grades. They demonstrated that academic and extracurricular achievement 
in college are not highly correlated and urged greater use of extracur- 
ricular achievement as an alternative criterion to grades in the develop- 
ment of selection devices. 

The present study grew out of these two sources of dissatisfaction with 
current methods of predicting success in college. This study has two main 
goals: First, to find non-intellective predictors that will make effective 



discriminations between students who have already been highly selected on test 
score; and second, to find predictors of extracurricular achievement which will, 
hopefully, be independent of the predictors of grades. 

Method 

Questionnaire materials were mailed to a sample of 1843 National Merit 
Finalists and National Merit Scholars in the spring of 1962, shortly before 
their expected graduation from high school, and also to a sample of 1383 stu- 
dents chosen at random from all students who took the National Merit Scholar- 
ship Qualifying Test (NMSQT) in 1961 the same time that this test was taken 

by the sample of Merit Finalists and Scholars. Usable returns were obtained 
from 86 $ of the Merit Finalists and Scholars (the Merit sample) and from 64 $ 
of the random sample of NMSQT participants (the normative sample). 

A second questionnaire was sent to the respondents to the first survey in 
the summer of the following year, when the students in the sample who attended 
college should nave finished their freshman year. This questionnaire inquired 
about their college grades, their extracurricular achievement and some additional 
items for another study. Returns were obtained from 92$ of the Merit sample and 
84 $ of the normative sample. Those who did not attend college for the entire 
year and those with missing data were discarded leaving a Merit sample of 1330 
and a normative sample of 419 for the study. 

Predictors 

The first questionnaire obtained student responses to four groups of per- 
sonality, attitude, interest and behavior items: (a) the California Psychologi- 
cal Inventory, a well known personality inventory consisting of 480 True -False 
items (Gough, 1957 ) j (b) the Vocational Preference Inventory (VPl), an interest 
inventory consisting of 160 occupational titles to which the student responds 
Like-Dislike (Holland, 195^); (c) the Adjective Check List (ACL), adapted with 



ERIC 



- * A .1 






4 



some additions and deletions from a checklist developed by Gough (i960) and in 
which the student checks from a list of 159 adjectives those he considers des- 
criptive of himself; and (d) the Objective Behavior Inventory (OBI), developed 
for this study and consisting of 326 things that a student might do (hobbies, 
sports, leisure time activities, interaction with others, etc.). The student 
indicates whether he has engaged in each activity frequently, occasionally or 
not at ail during the past year. In contrast to the other three inventories, 
which inquire about attitudes, opinions and feelings that are known only to 
the student himself, the OBI items refer to overt behavioral acts which could 
be seen by an external observer. 

Criteria 

Two criteria of college performance were used academic achievement and 

extracurricular achievement: (a) Academic achievement was assessed by the 
student's self report of his freshman grades on a letter grade scale (A, A-, 

B+, B, B-, etc.). Nichols and Holland (1963) compared such self reported 
grades with grades calculated from a transcript and found the self reports to 
be quite accurate (r=. 96 ). Davidsen (1963) found similar validity (r=. 92 ) for 
self-reported high school grades obtained in a selection context. 

The students attended a variety of colleges, and it seems likely that a 
given grade may mean quite different things in different colleges or even in 
different departments within a single university. To the extent that this is 
the case the grade criterion will be less predictable from student characteris- 
tics assessed before college entry. Thus, we would not expect to find validity 
coefficients as high as might be obtained when predicting them at a specific 
college. However, a sample of students attending many colleges offers two 
specific advantages: only predictors that have validity in a variety of col- 
lege environments will be found; and the validity coefficients will give an 
indication of the practical utility of the predictors in a setting where pre- 
dictions must be made before the student's college is known. 




(b) Extracurricular achievement was assessed from the student's report of his 
non-academic activities during the freshman year. The questionnaire asked the 
student to describe all achievements during the past year in the following areas: 
Leadership, Science, Art, Music, Writing, Speaking, IJramatics, and Athletics. 
Examples were given of possible achievements in each area and space was pro- 
vided for the student to write in his achievements. This free response pro- 
cedure is felt to be an improvement over the checklists used in previous 

studies (Nichols and Holland, 1963; Holland and Nichols, 1964) the possibility 

of error from random and erronious checking of checklists of very rare events 

was reduced, and many achievements were obtained that had not been included on 
the checklists. 

The responses in each area of achievement were coded on a three point scale: 
(l) no achievement, (2) achievement not involving outside recognition, (3) out- 
standing achievement receiving outside recognition of quality. These achieve- 
ments were combined into a single three point extracurricular criterion scale 
by counting those students with no achievement in any area as non-achievers 
(scored l), those with one or more outstanding achievement as achievers (scored 
3), and the rest in between (scored 2). A few students with recognized leader- 
ship achievement, but no other achievement, were scored 2 instead of 3 to keep 
the criterion scale from being too heavily weighted with leadership. The 
extracurricular achievements were combined into a single scale because there 
were not enough outstanding achievers in the individual areas and the lower 
level achievements were not of sufficient social significance to warrant 
separate study. However, combining the achievements is justifiable on both 
empirical and logical grounds: achievements in the various areas tend to have 
low positive correlations with each other. Since time limitations make it 
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difficult for a student to achieve in more than one area, the low positive 
relations seem to indicate a substantial general tendency to achieve. Roberts 
(1965) developed scales to predict specific extracurricular achievements and 
found similar content among the scales. His scales developed to predict achieve- 
ment in one area also predicted achievement in other areas, and his achievement 
scales had much higher intercorrelations among themselves than did the act ual 
achievements. Potential users of predictors of extracurricular achievement 
are more concerned with a general tendency to achieve than with the specific 
1 area of achievement, because, like grades, extracurricular achievements in col- 

lege are of value mainly as a demonstration of a tendency to achieve which is 
expected to persist to some degree after college. 

Analyses 

Scales were derived by item analysis against grades and extra-curricular 
! achievement using a portion of the Merit sample. After excluding students who 

did not attend college for a full year and those with incomplete data, 1013 
j Merit students (those with ID numbers ending with a digit less than seven) were 

assigned to the derivation group and the remaining 4l9 to the cross-validation 
group. No special separation of the sexes was made. 

! Por item ana lysxs against the grade criterion high and low groups of ap- 

: proximately the upper and lower 27# on first year college grades were used: 

the 262 students who reported a first year grade average of A or A- and also 
reported receiving some recognition for academic achievement (dean's list, honor 
society, etc.) composed the high group. The low group consisted of the 246 
students with a first year grade average of B- or lower and who received no 
recognition for academic achievement. Por item analysis against the extra- 
curricular achievement criterion, the high group consisted of the 229 students 
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who received outside recognition for one or more extracurricular achievements 
and the low group consisted of the 294 non -achievers. 

Phi coefficients were computed for each of the 1125 items against each 
criterion. The OBI items, which used a three alternative response format, 
were dichotomized by combining the middle category with the smallest extreme 
category. Separate academic and extracurricular achievement scales for each 
of the four item formats were formed by combining all items significant at 
the .05 level. The validity of these scales and their usefulness in combination 
with test scores and high school rank were checked using the cross-validation 
group of Merit students and the normative sample. 

Results 

The characteristics of the scales derived from the four item pools 1 are 
shown in Table 1. Of the major item types the OBI and the ACL had the largest 
proportion of significant items and the CPI the smallest proportion. The OBI 
scales had lower internal consistency than the scales from the other three item 
pools. These data are not sufficient evidence to establish the superiority of 
one item pool over another. The validity coefficients of scales composed from 
the various item pools are the most important consideration, and these are 
discussed later. 

Item Content 

The item content of the scales gives an indication of the kind of person 
who is likely to achieve) although it is important to remember that the items 
are self reports and that they were identified by group comparisons so that 
all items do not necessarily apply to all students. With these qualifications 
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1. The items and scoring keys for the scales developed in this study are 
available to responsible persons from the author on request. 
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Table 1 

Characteristics of Scales Derived from Four Item Pools 



T hpm Vnnl t\t. -i 






Grade 


Scale 






Achievement 


Scale 


iMuuiuer oi 

items 
in pool 


si<mir. items 
(P'"-05) 

w io Mean 


SD KR- 


Signif . 
(p<.05) 

21 


items 

a 

Mean 


SD ICR-21 


Objective Behavior 
Inventory (OBl) 


326 


107 


32.8 


63.5 


11.2 .80 


107 


32.8 


50.7 


n.i .79 


Adjective Check List (ACL) 


159 


50 


31.4 


31.2 


8.1 .84 


54 


34.0 


25.2 


9.2 .86 


Vocational Preference In- 


ventory (VPl) 


160 


35 


21.9 


25-3 


5-8 .82 


52 


32.5 


22.3 


9-6 .88 


California Psychological 


Inventory (CPl) 


480 


no 


22.9 


69.7 


12.1 .83 


97 


20.8 


57.6 


12.4 .86 



a ’ Deviations and Reliability Coefficients were calculated from Meri- 

s -validation sample, both sexes combined (N=302), 
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in mind the items significantly related to each criterion were grouped into 
what seemed to be homogenous content categories as follows: 

Content of the grade scales . The high grade group, when contrasted 
with the low grade group, more frequently endorsed items suggesting that 
they are religious and involved in church activities; are interested in 
school and value school work highly; are interested in music and partici- 
pate in musical activities; are hard workers with well established work 
habits; and are shy, socially withdrawn, and introverted. The low grade 
group on the other hand more frequently endorsed items suggesting that 
they are somewhat mischievous and thrill-seeking; are likely to engage 
in a long list of activities usually considered "fun" for young people; 
are erratic and undependable; and are interested in occupations which might 
be considered exciting or dangerous. 

The following adjectives were selected as representative of the ACL 
items significantly differentiating the high and low grade groups: High 
grade students described themselves as ambitious, capable, conscientious, 
dependable, efficient, helpful, methodical, modest, patient, quiet, re- 
sourceful, self-confident, timid, well-adjusted, and withdrawn. Low grade 
students described themselves as boastful, carefree, careless, cynical, 
disorderly, high-strung, impulsive, irresponsible, lazy, messy, rebellious, 
and sophisticated. 

A few individual CPI items deserve mention because they reinforce the 
general impression from the foregoing discussion that the student who gets 
good grades is likely to be compulsive and conforming. For example, the 
high grade group more frequently responded "True” to the following items: 

I am stricter about right and wrong than most people. I would disapprove 
of anyone's drinking to the point of intoxication at a party. I keep out 
of trouble at all costs. I consider a matter from every standpoint before 
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making a decision. I always like to keep my things neat and tidy and in 
good order. 

Content of the extra-curricular achievement scales. The group of 
achievers, when contrasted with the non-achievers, more frequently endorsed 
items suggesting that they engage in a variety of extracurricular acti- 
vities including music, speech, drama, science, art, etc. (all activities 
similar to the criterion achievements); are religious and involved in 
church activities; are outgoing and dominant in interpersonal situations; 
date frequently; are ambitious and hard working; and are interested in 
a variety of artistic and intellectual occupations. The non-achievers on 
the other hand more frequently endorsed items suggesting that they have 
a low energy level and are overly sensitive to the opinions of others. 

The following adjectives were selected as representative of the ACL 
items significantly differentiating the achieving and non-achieving groups: 
Achievers described themselves as aggressive, alert, ambitious, artistic, 
attractive, clever, confident, cooperative, deliberate, dominant, ego- 
tistical, energetic, generous, helpful, independent, ingenious, mature, 
original, persistent, responsible, sophisticated, unconventional, versa- 
tile, well thought of, and witty. Non -achievers described themselves 
as lazy, quiet, shy, slow, and unambitious. 

Cross-validation 

The intercorrelations of the various scales, test scores, and high school 
rank (HSR) are shown for the Merit cross-validation group in Table 2 and for 
the normative group in Table 3 . 

The correlations shown in these two tables lead to the following con- 
clusions: 
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Note: Decimals omitted 



(a) The two criteria had low positive correlations with each other for 

both sexes and both samples. These correlations are low enough for the grade 
and extracurricular criteria to be considered as independent for all 
practical purposes. 

(b) Among the non -intellective predictors, the CPI scale was the best 
predictor of grades (except for Merit girls) and the OBI scale was the 
best predictor of extracurricular achievement. For both criteria the 
CPI and OBI scales were substantially better predictors (average validity 
. 23 ) than were the ACL and VPI scales (average validity .11). 

(c) The two criteria were differentially predictable by the non-intellec- 
tive scales. In only five of 32 instances did a scale fail to correlate 
higher with the criterion for which it was derived than it did with the 
other criterion, and none of these five instances involved the CPI oh OBI. 

The scales for the two criteria were negatively correlated in the OBI and 
VPI and positively correlated in the CPI and ACL. 

(d) In the Merit sample the NMSQT and SAT had slight predictive validity 
for both criteria for boys and no validity for either criterion for girls, 
however, low correlations are to be expected in this instance because of 
restriction of range of test scores in the Merit sample. In the nor ma tive 
sample the NMSQT had some validity for both criteria for boys but was re- 
lated only to the grade criterion for girls. 

(e) HSR was related to the grade criterion for both sexes and both samples, 
but not to the extracurricular achievement criterion. It has been suggested 
that rank in high school class uncorrected for class size is as effective 

a predictor of college grades as is percentile rank. This was true for 
boys in both samples, but not for girls. 
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(f) Two reports of high school rank were available for the Merit sample, 
one from the school and one from the student. The correlation between 
these two reports was .8l for boys and .82 for girls. This is a somewhat 
lower validity for self reported grades than others have reported. A 
possible reason is that in this study students were asked to report both 
rank and class size, from which percentile rank was computed. This task 
may have been more difficult than the usual report of a grade average. 

The restricted range of class ranks among the Meri b students is another 
possible reason. Whatever the source of error in the self reported HSR 
may be it is indeed a peculiar kind of error: the correlation with college 
grades for girls is the same for self reported HSR as for school reported 
HSR, but for boys self reported HSR has a substantially, but not signi- 
ficantly, higher correlation with college grades than does school reported 
HSR. 

(g) it is possible to form a rough ordering of the three classes of pre- 
dictors (test scores, non-intellective scales, and HSR) in terms of their 
validity for predicting the two criteria, and this order holds in general 
for both samples and both sexes. To give an indication of the relative 
size of the coefficients we will use the average validity of the OBI and 
CPI scales to represent the non-intellective scales, the NMSQT Composite 
to represent the aptitude tests, and percentile rank to represent HSR. 

For predicting college grades, high school grades were the best predictor 
(average validity .33), followed by the non-intellective scales (average 
validity .27), and finally by the aptitude test (average validity .12). 

For predicting extracurricular achievement the non-intellective scales 
were the best predictor (average validity .19), followed by high school 
















grades (average validity .10), and the aptitude test (average validity .07). 

Grades were predicted with higher validities by all three classes of pre- 
dictors than was extracurricular achievement. 

(h) Girls tended to achieve at a higher level than boys in most respects. 

They made better grades in high school and college, they obtained higher 
scores in both groups on non-intellective scales, and they obtained 
higher verbal test scores. There was no sex difference in extra- curricular 
achievement, and boys obtained higher scores on quantitative tests. The 
achievement of boys was somewhat more predictable than that of girls: 
the non-intellective scales had higher validities for boys and in the 

I- 

I 

Merit sample the test scores also had higher validities for boys. 

Multiple Correlations 

A number of multiple regression equations were computed for predicting 
each criterion from various combinations of predictors in the various samples. 

Since these equations showed little consistency from one sample to another 
and since the multiple correlation coefficients are subject to an unknown 
amount of shrinkage, they will not be reported in detail here. However, a 
few general patterns emerged consistently from the various analyses: 

(a) The non-intellective scales derived from the four item pools did not con- 
tribute unique information. Once the scale with the highest correlation with 
the criterion was taken into account the partial correlations of the other 
scales with the criterion were non -signifi cant . (b) The non-intellective scales ■ 

added significantly in most instances to the prediction of the grade criterion 
that was possible with HSR and the SAT or NMSQT Composite, (c) HSR and the 
test scores in most instances did not add significantly to the prediction of 
extracurricular achievement that was possible with a single non-intellective 
scale. s 
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Discussion 



The scales to predict grades developed from the OBI and CPI pools add sig- 
nificantly to the prediction of college grades that is possible from test scores 
and high school rank. The extracurricular achievement scales seem to be the best 
available predictors of extracurricular achievement. Moreover, these scales 
appear to be particularly robust: developed on a sample of very able students of 
both sexes, they maintained their validity when applied to boys and girls sep- 
arately and when applied to a sample of students considerably lower on the ability 
scale than the sample on which they were derived. 



These results warrant tentative use and further trial of these scales In 

2 



actual selection situations. If the selection context does not greatly af- 
fect their validity, the scales will provide an increment in the accuracy of 
prediction of college performance. 

Extracurricular achievement in this study was not as predictable as were 
grades. Moreover, the predictors of extracurricular achievement were primarily 
high school activities and extraversion— traits which may not be considered valuable 
m themselves. This raises the question of what weight should be given in col- 
lege and scholarship selection programs to predictors of extracurricular achieve- 
ment. It is conceivable that some admissions officers may wish to select 
potential extracurricular achievers to promote campus activities or to balan ce 
an overly studious student body. However, in most selection programs the justi- 
fication for selecting potential extracurricular achievers must be based on the 
assumption that such achievers will make socially valuable achievements after 
college. Although this assumption seems reasonable enough, there is little 
good evidence either to support or to refute it. 

There is a clear need for better criteria of success in college, and the 
lack of good criteria is one of the main difficulties in improving predictors. 

Since the criterion value of any index of college achievement depends in large 
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^ Validity of the scales when used in the selection context of 
the National Achievement Scholarship Program for outstanding Negro students 

in +yl Tiie non -intellective scales may offer a special advantage 
in the NASP where the value of the traditional predictors may be attenuated 
by cultural factors. 
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part on its relationship to post college achievement, studies of the correlation 
of college behavior with socially significant achievement after college might 
help to clarify the criterion problem. 

A specific finding which deserves comment is the low predictive validity 
of the aptitude tests in this study. This might be expected in the Merit sample 
with its restricted range of scores, but it was also true in the normative 
sample. The correlations between NMSQT Composite and college grades in the 
normative sample of .18 for boys and .23 for girls are far below the correlations 
in the .50's reported in the NMSQT Manual (Science Research Associates, 1964 ) 
for students attending particular colleges. This shrinkage in validity is un- 
doubtedly due to the fact that the students in the present samples attended 
many different colleges, with the high scoring students attending colleges where 
the academic competition is greater than for the low scoring students. The 
shrinkage in validity for the test scores seems greater than for HSR (normative 
sample validities of .34 for boys and .38 for girls). The great differentiation 
of colleges on tested ability may effectively destroy the validity of test scores 
for predicting college grades in a heterogenous group of colleges, leaving the 
non-intellective factors, on which colleges are not so differentiated, as the 
best predictors. This reasoning would seem to indicate that those who deal 
with students attending a variety of colleges should develop ways of taking 
the college into account if they wish to use grades as an index of success. 

Validity is often not the sole value in a selection program, and the use 
of non-intellective scales raises some issues which, although always present, 
are hidden when intellectual predictors are used. Stalnaker (1965) stated the 
issue well when he said (prophetically), "In a program very much in the public 
eye, predictive validity alone cannot rule. . .Suppose there should develop 
sound evidence that among the highly intelligent, the most conforming, compulsive. 
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dependent, unoriginal individuals do best in college. Should we then try to 
limit our selection to students having these characteristics" (p. 135)? Stair 
naker's hypothetical example is only a slight exaggeration of the content of 
the non-intellective grade scales. The extracurricular scales on the surface 
would seem to select more the "All American Boy" type, but should a student be 
awarded a scholarship because of his broad interests and frequent dates? 

The personality traits of the selected students become explicit with the 
use of non-intellective predictors, but selection for personality characteristics 
is implicit in all selection programs. For example, unpublished comparisons 
of the Merit and normative samples used in this study reveal that the HMSQT 
tends to select a socially withdrawn, studious, introversive character. 

Since the composition of social groups in our society is increasingly deter- 
mined by centralized selection programs, more attention should be given to 
the type of person identified by the various selection strategies. The ex- 
plicit recognition of the role o£ personality traits that is inherent in the 
use of non-intellective predictors may help focus attention on this problem. 
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